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Choice Reaction Time and Psychometric 
Performance Revisited 



Clifford E. Lunneborg 



As part of a long term project relating Individual differences In 
performance on paper and pencil tests of Intellectual aptitude and 
achievement to differences observed In laboratory studies of cognition, 
data have been collected con^arlng test scores with simple choice 
reaction times. The three studies summarized here suggest that the 
relationship may be fairly strong except where the saiiq>llng of subjects 
to be studied seriously restricts variability In those measures. Por- 
tions of these results were communicated In papers read by the author 
at the 1974 and 1975 conventions of the American Psychological 
Association. The research was supported by grants from the National 
Institute of Education to the author and from the National Institute 
of Mental Health to E. B. Hunt, Department of Psychology, University 
of Washington. 
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Both experimental cognitive psychology and psychometrlcs arci 
concerned with the development of structures which explain human Informa- 
tion processing. To the extent that they share this area of mutual 
concern It ought to be possible to establlBh some correspondence between 
measures of Individual performance alo:)g dimaiislons Implicit In modem 
theories of human cognition, particularly theories of memorial organiza- 
tion and operation, and measures of intelXigence or cognitive abilities. 
If such relations can be established tiiay would be of Interest both to 
the cognitive theorist and to the psychometrist. For cognitive psy- 
chology the message would be that individual differences in intelligence 
are well-established and naad to be mirrored by individual differences 
in parameters postulated for any cognitive theory. For the psychometrist 
the expectation is that assessment of purely normative individual dif- 
ferences in global performance can be replaced by individual differences 
which can be expressed in terms of a processing metric (time, capacity, 
rate) required by a theory of information processing. In short, research 
in this area is directed at providing either a theory of cognition that 
takes individual differences into account or a means of assessing 
individual differences within some theoretical basis. 

One theoretical basis for such bridge-building work is provided by 
the distributed memory model (Hunt 1971, 1973). Briefly characterized, 
the distributed memory model is concernad xdlth the buffering of seusory 
information into central memory, recodlng of information within and 
between memory stages, retrieval of information from memory, and the 
cross-referencing and coiipariscn of infcrr-cition stored in memory. Human 
information processing is likened to a computing system with control 
processes manipulating informatiorx stored in a data structure subject to 
the constraints of the brain's system architecture. Between individuals, 
differences in cognitive performance can be contributed to by differen- 
tial characteristics of the physical cystem, by differences in availa- 
bility or efficiency of control processes, or by differences in the 
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organization of the data structure. How might differences of these kinds 
relate to tested differences, for e:cample, in verbal intelligence or 
spatial ability? 

Differences in the speed with which individuals accomplish basic 
information processing tasks are a prime candidate for study and data 
from three studies are reported here which bear on the question of how 
such differences, measured in the laboratory, relate to performance on 
paper and pencil psychometric instruments, standard aptitude tests. 

Study 1 

Carroll (1974) worked with a representative sample of psychometric 
tests, those assembled in the Kit of Reference Tests for Cognitive Factors 
(French, Ekstrom, and Price, 1963), examining each with respect to the 
same distributed memory, information processing model mentioned earlier. 
This resulted in the characterization of each test with respect to the 
demands that test makes on an information processing system. More 
specifically, those information processing characteristics were identified 
which would be expected to show appreciable individual differences. 

Carroll's theoretical analysis suggested a framework within which to 
look at relations between laboratory measures of information processing 
and test performance. 

Method 

Subjects : Sixty-four high school age subjects from the Seattle area 
were recruited summer 1973 via a community Job Line. In groups of eight 
they spent half a day on each of two consecutive days in the laboratory. 
During each session they participated in several short laboratory assess- 
ments and completed a battery of short paper and pencil tests. Subjects 
were paid on an hourly basis for their participation. 

Psychometric Instruments ; The tests may be briefly described. 

1. Verbal Comprehension, EAS test 1, calls for the subject to select 
a synonym from among five alternatives for each of 30 words. Five minutes 
are allowed to work on the task. As with :he other multiple choice tests 
in this battery the score is the nunber of items answered correctly minus 
a fraction of those answered incorrectly. In the Reference Kit scheme 
this test measures Factor V, Verbal Comprehension. 
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2. Numerical Ability , EAS test 2. Part 1 contains 25 items of the 
form "16 + 18 = with the answers to be selected Jrom five alter- 
natives. Two minutes are allowed on this part. Part 2 consists of 25 

problems of decimal fractions, for example, "1.6 x 0.3 ^ Four 

minutes were allowed for this part. The two parts provided a single 
score. This test measures Factor Number Facility. 

3. Space Visualization , EAS test 5, presents the subject with a 
series of ten line drawings. Each drawing depicts a stack of blocks and 
the subject's task is to report, by marking on an answer sheet, for each 
of the labelled blocks the number of other blocks in the stack that are 
in contact with it. Five minutes are allowed and the score is the number 
of correct counts completed. The test is considered a measure of Factor 
Vz , Visualization . 

4. Numerical Reasoning , EAS test 6, presents subjects with twenty 
number series. The task is to select the next, eighth element from 
among five alternatives. 

This test taps Factor I, Induction, as well as Number Facility. 

5. Verbal Reasoning , EAS test 7, is a deductive reasoning test and 
a measure of cognitive factor Rs, Syllogistic Reasoning. Subjects are 
presented \rLth a set of four facts followed by five conclusions and asked 
to report for each conclusion whether it is true, false, or of indetermi- 
nate validity. Six such exercises may be attempted in the five minutes 
testing time. 

6 and 7. The Minnesota Clerical test consists of two parts each 
yielding a separate score. In part 1 subjects compare pairs of numbers 
(from 3 to 12 digits in length) and check the pairs which are the same 
(3 minutes). The second part of the test is identical except that the 
pairs consist of names of individuals or businesses, and again, this 
part runs for 3 minutes. These Clerical Number and Clerical Name 
scores should both tap Factor P, Perceptual Speed. 

8. Hidden Figures is an experimental test included in the 
Reference Kit as a measure of Factor Cf, Flexibility of Closure. Subjects 
are presented with five polygons and asked to determine for each of six- 
teen complex line drawings which polygon is hidden in the complex figure. 
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Laboratory Measures ; The laboratory sessions involved subjects in 
several settings. Only those which provided measures for the present 
analyses, however, will be described here. 

1. Motor Reaction Time was the median time taken to respond by key 
press to the onset of a on a CRT screen. 

2. A one-"handed choice reaction time study provided a median time 
to respond discriminativcly by depressing a left or right key to two 
stimuli, a^^ircle appearing either in the left or right half of a CRT 
display. ■"'"This choice reaction time was not itself employed in the present 
analysis. Rather a variable called Choice Time was defined as the differ- 
ence between the median choice and motor reaction times. 

3. The choice reaction time study above also provided a second 
variable for this analysis: proportion of trials on which the correct 
choice was made . 

4. In the Stroop task subjects were asked first to report orally 
the colors in which a series of asterisks were printed and then report 
the color of printing of an equivalent length list of contrasting color , 
names • The task was repeated twice and the score used here x^as the 
average difference in '' reading *'" times between the name and asterisk 
conditions 

5 and 6. In an experiment in the Sternberg (1970) paradigm S^s were 
sequentially shown from one to six consonants and then shown a single 
probe consonant and asked whether the probe was in the previously exposed 
set. Response time was recorded and, for correctly identified instances, 
these times linearly regressed on the number of digits in the associated 
target set. For each S^, then, a slope and intercept value were obtained. 

7 and 8. Strings of fifteen digits were presented binaurally to Ss 
with recall cued immediately following presentation. In recall S^s were 
instructed to recall in order as many digits as possible beginning with 
the first digit heard. Ten trials were presented with each scored for 
the number of digits recalled in order. In this analysis two scores were 
employed — a Digit Span ^ Final score (average of performances on the last 
five trials) and Digit Span , Gain score (difference between the Final 
average and the average over the first five trials) . 



6 



9 and 10. In a task patterned after Massaro (1972) four digits and 
four consonants were presented dicUo tic ally, two digits and two conso- 
nants to each ear. Following presentation S^s were cued to recall the 
presented material. On some blocks of trials S^s were to report by ear, 
i.e., report what they heard to the right ear or the left ear. On other 
blocks they were asked to report by category, either digits or consonants. 
While they knew whether they were going to have to recall by category or 
by ear, which category or which ear was not cued until after the presen- 
tation. Two scores were extracted for the present analysis. The 
Category Score gives the number of items correctly reported over all 40^ 
category trials. An Ear Minus Category Score was obtained by subtracting 
the category score from a similar score obtained from the ear trials. 

11-14. A final set of four scores was obtained from a study of 
clustering in recall of a list of nouns based on the experimental para- 
digm of Puff (1966) . During their first laboratory session ^s were shown 
item by item, two lists of 30 common nouns. Each list was shown twice 
with subjects asked for recall immediately following each presentation 
(four recalls) . Each list consisted of ten nouns each from three se- 
mantic categories (fruits, occupations, animals, etc.). For half the 
subjects the list presented first was blocked, all ten members of a 
given category appearing contiguously, and the list presented second was 
in pseudo-random order. For the other half of the Ss this order was 
reversed. When S^s returned for thJlr second day in the laboratory they 
were asked to recall each of the two lists (two more recalls). A clus- 
tering score was computed for each of these six recalled lists. In the 
present analysis four transformed s res were employed: (11) The clus- 
tering immediately following the second presentation of the blocked list 
provided a base score: Bl; (12) this base clustering minus the clus- 
tering for second presentation of pseudo-random list: Bl-Rl ; and (13) 
base clustering minus the clustering for second day recall of blocked 
list: B1-B2 ; and (14) this second day blocked clustering score minus 
the second day random clustering: B2--R2- 



Results 

Correlations between the pcychometric and laboratory data are shown 
in Table 1. The response time measures — Motor R T, Choice Time, the 
Stroop measure, and the two scoras from the Sternberg task — correlated 
negatively with paper and pencil test performance, as might be expected. 
Proportion of correct responses in the choice reaction time task corre- 
lated only vreakly, though positively, with the psychometric measures. 
This laboratory raeasure had little variability, however, as few errors 
were made (average proportion correct was .93). The negative correla- 
tions involving Digit Span, Geiu suggest the possibility that.^as 
improvement on the average was small (from 5.75 digits to 6.23 di'gits), 
big gains were registered by those who bad not done well in the initial 
trials. The digit span score itself was not highly correlated with any 
of the psychometric measures. 

The Category Scores from the dichotic listening task correlated 
positively with nearly all of the paper and pencil tests. Raw ear scores 
were considerably smaller than category scores (roughly 70 as opposed to 
90 on the average). The pattern of correlation for the Ear Minus Cate- 
gory scores suggests that highest psychometric scores were earned by S^s 
whose category performance far outstripped their ear performance. 
Finally, the Clustering Score is such that low scores (actually large 
negative scores) indicate greater clustered recall. Except- for the 
reversal of signs the pattern of correlations for the clustering on 
immediate recall of the blocked list (Bl) is quite similar to that for 
the Category Score from the dichotic listening task. The difference in 
clustering scores between the immediate recall of blocked and randomized 
lists (Bl-Rl) was not well correlated with any of the psychometric 
measures. (Clustering was, of course, greater for the blocked than for 
the random list.) Clustering for the blocked list was almost exactly 
the same for delayed as for immediate recall (the score B1--B2 had a mean 
close to zero). This difference, however, did correlate with a number 
of psychometric measures. Increased reliance on semantic clustering 
between immediate and delayed recall tended to be negatively related to 
certain paper and pencil test performances. The last measure included 
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in Table 1, B2-R2, the difference in clustering of delayed recall between 
the blocked and randomized lists^had near zero correlations with psycho- 
metric measures. (For the randomized list, however, clustering was con* 
y siderably greater on the average for delayed than for immediate recall.) 
Table 1 reports zero order correlations between laboratory and 
psychometric measures. Because many laboratory measures are themselves 
not pure measures of information processing parameters — for example, 
clustering score Bl may be interpreted as a measure of short-term serial 
recall ability as well as any semantic clustering tendency — these corre- 
lations are more equivocal than desired. The laboratory perform- 
ances were correlated among thamselves. This is reflected in Table 2. 
The quickness of response measures — 1, 2, 4, 5, and 6 — tended to be 
positively correlated. Clustering on immediate recall of a blocked list 
had a sizeable negative correlation (^55) with the category score in the 
dichotic listening task. Tables 1 and 2 provide the essential data for 
the generation of the final results to be reported. 

The systematic stepwise multiple regression analyses completed provide 
a quantitative analog to Carroll's (1974) point about the information 
processing or memorial complexity of most paper and pencil tests. The 
strategy was to order the laboratory measures by level of complexity 
ranging from the motor reaction time measure, which places the least load 
on memorial operations, to recall measures in the dichotic listening and 
clustering studies. The multiple regression analyses assessed how much 
of the inter-individual variability in performance on each of the paper 
and pencil tests could be accounted for successively by the independent 
contributions of the several laboratory measures, working up from the 

least complex. The results are given in Table 3. Entries in the columns 
2 

headed R are cumulative proportions of variance accounted for. Motor 
Reaction Time accounted for 16% of the variance in the Verbal Coiiq)rehen- 
sion scores; Motor Reaction Time plus Choice Time accounted for 22% of 
the variance, etc. The A entries are the increments, the additional 
proportion of variance accounted for. It is important to keep in mind 
that these increments are for partial variables. The contribution of the 
Choice Time variable, for example, is the contribution of that measure 
after it has had partialled from it the influence of Motor Reaction Time. 
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Figure 1 is a simplication of the data in Table 3. The measures 
have been grouped as shown by the letters along the right edge of Table 
3. ThuS;, A indicates the variance accounted for by the Motor Reaction 
Time; B is a measure of the amount of additional variance accounted for 
by the remaining response time measures, C is the contribution of a 
caref ulness-of-response lueasure — proportion of correct responses in the 
simple, two-choice reaction time study; D carries the incremental con- 
tribution of the digit span study; E does the same for the dichotic 
listening task, and F and G bring in^ finally, any remaining contribution 
of the clustering on recall scores. 

Discussion 

Before discussing Figure 1, using Carroll ^s (1974) characterization r 
of what is important to doing well on measures of the several cognitive 
factors, it is important to note as he did that the time limit on a test 
introduces a speed component to test scores of some magnitude. Because 
Carroll addressed himself to the question of what is important in re- 
sponding to an item rather than to a speeded test as a whole, it might be 
expected that a generalized measure of response time would be a more signif 
icant determiner of total test performance than of success with a single 
item. That is, the component labelled '^A*' in Figure 1 may be inflated. 

Space Visualization . Carroll characterized Factor Vz as dependent 
on a process occurring in STM whereby a spatial representation is 
''mentally'' transformed. This process is assumed to produce both capacity 
and rate differences between individuals. Nothing in the present labora- 
tory measures speaks to the question^of capacity for spatial representation 
However, the rate at which this representation can be manipulated may well 
not be unique but dependent upon more general STM processing rates. This 
would be consistent with the appreciable individual differences accounted 
for by the time components A and B. 

Numerical Ability . Factor N requires retrieving number associations 
and arithmetic algorithms from LTM and performing serial operations on the 
stimulus materials using the retrieved algorithm. Individual differences, 
Carroll noted, appear in both content and temporal aspects of the re- 
trieval and manipulative operations. Carroll also felt that special 
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strategies — chunking numbers, for example—may also contribute to 
individual differences. Rate, either of retrieval of an algorithm and/or 
of application of the algorithm was apparentl}^ important and, again, could 
be determined by more general processing rates assessed in A and B. 

Numerical Reasoning . The number series problems were most akin to 
Factor Induction, suggested by Carroll to require searching a general 
logic store in LTM for relevant hypotheses. The major determiner of 
success is whether the contents hold the solution. Some subjects, 
however, might construct*, new hypotheses by serially operating upon STM 
contents. A substantial amount of the variability in this test, however^ 
was accounted for with no knowledge of the contents of an LTM logic store. 
Perhaps the range of inductive hypotheses likely to be employed is 
limited enough by the sampling of problems and subjects as to not be a 
source of individual differences. The importance of speed, again, plus 
the relatively heavy contribution of the dichotic listening and clustering 
scores suggests that operations on STM were indeed contributory to success. 

Verbal Reasoning . Representative of Factor Rs, Syllogistic 
Reasoning, this test would require by Carroll's analysis retrieval of 
meanings and algorithms from LTM and the performance of serial operations 
in STM on the retrieved materials. Individual differences in content and 
temporal aspects of performance are postulated. There is some likelihood, 
as well, that _Ss vary in attention paid to details of the stimulus mate- 
rial. Results for Verbal Reasoning are in interesting contrast to those 
for Numerical Reasoning. That only a relatively small proportion of the 
total variance was accounted for, together with a depressed contribution 
of the processing rate measures, suggest that indeed it is individual 
variability in the contents of some LTM store (rather than the speed with 
which it can be accessed) that accounted for individual differences ia 
performance. An alternative explanation would be individual variability 
in performing transformations of statements prior to evaluating their 
logical consistency. If E, F, and G measure a general class of such 
transformations, then their small contributions to Verbal Reasoning argue 
against this alternative. Finally, if C, the contribution of carefulness- 
of- response, is interpreted as "attention to detail" (appealing in connec- 
tion with the next two tests), then ''attention to detail" was not an 
important source of variability in Verbal Reasoning. 

Er|c 15 
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Clerical , Number and Mame , Both subtests tap Factor P, Perceptual 
Speed, which should be primarily sensitive to the temporal parameters of 
a visual search through a fieldfor specified elements. This search, 
Carroll posited, occurs by addressing sensory buffers. Failure to 
account :or more of the variance in these two tests cannot be laid to 
failure to tap some LTM store. It is, in fact, puzzling that there is 
so much left unaccounted for. The inference is that the speed of proc- 
essing required in clerical work was not well explained by the speed of 
processing measures tapped by A and B. That C makes its greatest con- 
tribution here is suggestive of Important individual differences in some 
kind of testing or checking loop. 

Verbal Comprehension . This test and Factor V relate, roughly, to 
size of vocabulary and Carroll nas written that performance is almost 
exclusively dependent upon the contents of a lexicosemantic LTI^ store, 
upon the probability that can retrieve the correct meaning of a word. 
Figure 1, by contrast, reflects what has been found in other recent studies 
(Hunt, Lunneborg, and Lewis, 1975), that differences in verbal ability can 
be largely accounted for without ever inquiring about what S^s may know 
about the language. Being able to account for this variability for a 
group of adolescent native English speakers may not meet all of the goals 
of knowing what an individual S^'s vocabulary is. The Vocabulary test 
example, however, draws rather nicely a distinction between aptitude and 
achievement measures. The size of one's vocabulary is one measure of that 
individual's achievement. That it is so predictable from measures which 
are, from an information processing point of view, more basic than inven- 
torying the contents of an LTM store suggests that vocabulary might be 
supplanted as an aptitude measure, however. 

Hidden Figures . This last test is a measure of Factor Cf, 
Flexibility of Closure, and involves, by Carroll's analysis, an STM proc- 
ess whereby a figure is imaged in relation to a surrounding visual-- 
representational field. Both capacity and temporal aspects may be 
involved. The test was, for this samples a very difficult one — the aver- 
age score was 3.5 out of a possible 16. This is reflected in the rela- 
tively small contribution of the motor reaction time measure — so few 
responses were made by the average that speed of making the response 
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was unimportant. Choice Time was Important » as It was for the Spatial 
Visualization and Numerical Reasoning tasks , suggesting that successful 
test performance requires rapid evaluation of alternatives. Because of 
the purely flgural nature of the stimulus material It Is Interesting to 
note the rather large contributions of £ and measures grounded In symbolic 
and semantic manipulations. The suggestion here Is that the ability to 
effect these manipulations was not Independent of the ability to manipulate 
flgural material. 

Despite the shortcomings of the available data these analyses suggest 
that Carroll' s thoughtful analysis of the demands psychometric tests place 
on human beings as information processors can be fruitfully linked to 
laboratory measures to gain better understanding of what the psychome- 
trlsts call intelligence and what the experimental psychologist calls 
cognition. Interestingly, quite basic measures of individual differences 
in information processing — 

motor and choice reaction times — appeared to be strongly predictive of 
individual differences in test performance across a range of content. 

Study 2 

The rather consistent relations found in this first study were 
surprisingly strong and igepllcatlon, if not extension, of the results was 
in order. In this second study, the psychometric Instruments were selected 
to provide a more systematic (and psychometrlcally reliable) coverage of 
the cognitive domain, including, in particular, better coverage of those 
cognitive factors which Carroll (1974) depicted as making the more minimal 
demands of the information processing system; tasks which in the distribu- 
ted memory model focus on the sensory register and short term memory. At 
the same time, the laboratory measures of choice response time were adjusted 
to provide more reliable assessments of both practiced and unpracticed 
performance. 

Method 

Subjects . University freshmen were recruited fall 1974 from those who had 
completed the battery of tests administered high school juniors spring 19731y the 
Washington Pre-College (WPC) Testing Program. Volunteers were selected so 
that the joint distribution of the WPC Verbal and Quantitative Composite 
scores in the sample reflected the distribution for the entire entering class. 
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S^s were paid and their participation, as regards the data reported 
here 5 extended over an academic quarter. Typically, the would be 
involved in one psychometric and one laboratory session each week. The 
total group reported on here consisted of 63 freshmen, 27 men and 3S women. 

Psychometric instruments . Those instruraents contributing to the 
present study can be grouped under conventional psychometric headings. 
Because of tire nunber of tests involved a preliminary analysis of measures 
under each heading was undertaken to reduce the data utilized in later 
analyses . 

1 and 2. V erbal ability m easures r The WPC battery administered in . 
high school provided scores from tests of Readin,^ Comprehension , Vocabulary , 
Spelling , and English Usage (or , Graminar) . (The Verbal Composite score 
used in subject selection is a linear v/eighting of these four tests.) Ver- 
bal tests administered following subject selection included the EAS test 
of Verbal Comprehension used in Study 1 and^the'two subparts of the Terraan 
Concept Mastery test'. Synonyms and Antonyms and Verbal Analogies . In the 
French (1963) scheme all tests would be considered measures of Factor V: 
Verbal Comprehension. 

To reduce the number of verbal measures to be treated here a 
principal components analysis of the correlations among these measures 
followed by a varimax rotation of the larger of these components suggested 
a red\iction to two measures: 

(L) VOCABULARY - Average of standardized WC Vocabulary and 
Tennan Synonym- Antonym scores; 
and 

(2) GRAl^MAR - Average of standardized WPC Spelling and English 
Usage scores. 

3. Quantitative ability measures . The WC battery provided three 
quantitative subtests: Quantitative Skills (numeric reasoning). Applied 
Mathematics (word problems at the elementary algebra level) and Mathematics 
Achievement (achievement in high school mathematics) « The three are 
linearly combined to provide the Quantitative Composite measure used in 
subject selection. Additional quantitative tests administered this sample 
included the EAS tests of Numerical Ability and Numerical Rd rasoning 
described for Study 1. One principal component appeared to tap the 



18 



17 

variability central to all these measures, the Number Facility (N) factor 
being probably more important than the Reasoning (R) factor sometimes asso- 
ciated with certain of these subtests, and in the present analyses only a 
single quantitative measure was employed; 

0) QUAi>ITITATIVE C0I1P0SITE - the OTC overall quantitative score. 

4 and 5. Spatial ability measures. There was a single Spatial Ability 
measure included in the WPC. It is a paper folding test of the kind clas- 
sified by French (1963) as a measure of the Visualization (Vz) factor. 
Two other Visualization measures were administered, the Two Dimensional 
and Three Dimensional Spatial R elations tests of the California Mental 
Abilities Test (CMT) . The two-dimensional test is a paper form board- 
determining which pieces fit together to form a desired plane figure — while 
the three-dimensional test is one of surface development — determining which 
planar figure can be folded into the desired three dimensiona.l formr 

Other figural tests included the Hidden Figures test xrMch vas used 
in Study 1 and loads the Flexibility of Closure (Cf) and, to a lesser 
degree^ the Spatial Orientation (S) factor as well; a Maze Tracing task 
associated with a Spatial Scanning (Ss) factor; and the Advanced Progres- 
sive Matrices ( Set II ) of Raven. This last test is variously considered 
a measure of general intelligence Cg*\ and culture free at that), or an 
indicator of a reasoning factor styled Induction (I), or a "space'' mea- 
sure. The task is to pick a figure to substitute for a missing part of a 
larger figure so as to ''best complete the design/' It is included here 
because in the present study it had sizeable correlations with other space 
measures. 

The principal components/varimax rotation analyses of these figural 
or spatial tests suggested two clusters and — for this report — two new 
composite measures were defined: 

(4) SPACE ONE - Average of standardized Maze Tracing and paper form 
board test scores; and 

(5) SPACE TWO - Average of standardized Progressive Matrices, Hidden 
Figures , surface development, and paper folding scores. 

6 and 7. Reasoning ability tests . Two other of the short EAS tests 
were administered — Verbal Reasoning , used in Study 1 as a measure of the 
Syllogistic Reasoning (Rr) factor^ and Symbolic Reasoning . The last test 
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involves the concepts — presented using the customary symbols — of equality 
and inequality and poses questions of the form "if X £ Y and Y Z then 
is it true that X = Z?" It appeared to also sample the Ra factor. 

The Watson-Glaser .Critical Thinking Analysis was also administered 
and scores obtained for the five subtests — Inference , Recognition of 
Assumptions , Deduction, Interpretation , and Evaluation of Arguments . With 
the possible exception of Deduction these subtests are probably factorially 
complex — calling for a comprehension of material read and its integration 
with some already known information as?; well as the excercise of reasoning. 

When interrelations among the tests were analyzed the Symbolic 
Reasonin g test was more strongly related to the quantitative measures while 
the EAS Verbal Reasoning and three of the Watson-Glaser subtests — 
Recognition of Assumptions , Deduction ^ and I nterpretation — were moderately 
correlated with one or another of the verbal ability measures. In conse- 
quence, only two reasoning measures were retained: 

(6) INFERENCE - On the basis of a paragraph read, S^s classify a 

series of statements as true, probably true, false, probably 

false or based on insufficient data; 

and 

' (7) EVALUATION OF ARGUMENTS - a "policy question" is followed by 

a number of statements pro and con to be classified as vreak or 
strong arguments on the question. 
8» 9, and 10. Perceptual / motor speed tests . A number of tests were 
administered S^s which were highly speeded, placing a premium on quick 
perception and response. The Survey of Working Speed and Accuracy, for 
example, has four five-minute sections: Number Checking , comparing pairs 
of numbers for identity; Code Translation , substituting letters for 
integers; Finger Dexterity ^ dotting small circles; and Vowel Cou n ting , 
obtaining separate counts of A, E, I^ 0, and U occurrences in each line 
of text. Number Checking > Vowel Counting ^ and possibly the code substitu- 
tion task are measures of the Perceptual Speed (P) factor. Finger Dex- 
terity has been regarded as motor rather than cognitive. 

^ Code Recognition test was administered from the Flanagan Aptitude 
Classification battery. S^s were given practice in learning alphanumeric 
codes for common objects and then tested immediately on their recognition 
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of these codes. The test is a recognition form of the measuifes used as 
markers for the Associative Memory (Ma) factor. 

Finally, three locally constructed tests were administered, based upon 
the description given by Rose (1974) of a battery developed at the Human 
Performance Center, University of Michigan. The three were the GramtTiAtlcal 
Reasoning test, requiring Ss to compare a statement '^A does not precede B," 
for example, with a letter pair A3 or BA and decide whether the statement 
is true or false; the Neisser Letter Search test, requiring ^s to check 
off blocks of "randomly^* assembled letters that contain a key letter (in 
successive repetitions ^s search for one, two, and four letters); and the 
Fitts Tapping test in which _Ss place pencil dots within circular targets 
whose size and separation distance are systematically varied. 

Letter Search should be another Perceptual Speed test. Grammatical 
Reasoning was used as a paper and pencil surrogate for a laboratory pro- 
cedure used in earlier studies (Hunt, Lunneborg, and Lewis, 1975). Simi- 
larly, the tapping task was intended to provide a paper and pencil measure 
of a fairly xmencumbered response speed. 

Because the elements of this "Michigan battery" were so simple they 
were administered under highly speeded conditions. The battery was admin- 
istered twice on each of two occasions separated by about three weeks. 
(The two administrations on any one occasion included two logically 
parallel forms of the Grammatical Reasoning and Letter Search tests and 
two exact repetitions of the tapping tasks.) 

Upon analysis, the between sessions correlations for the same test 
were disappointingly low while the between test correlations for the final 
session were all in excess of .90. As a representative of the "Michigan 
Battery" one score was retained: 

(8) LETTER SEARCH - Average of standardized scores for the searches 
for 1, 2, and 4 letters on the fourth administration. 

An. analysis of the correlations among the other perceptual/motor 
test scores suggested three dimensions marked in turn by Number Checking , 
Code Translation , and Vowel Counting . As V owel Counting was strongly 
correlated with Letter Search only the first two scores were kept for the 
present analyses: 
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(9) NUl^IBER CHECKING - Comparing pairs of numbers 
and 

(10) CODE SUBSTITUTION - Replacing integers with arbitrarily paired 
letters. 

Of the ten psychometric scores, six were based on essentially 
unspeeded performance while four, LETTER SEARCH, NUMBER CHECKING, CODE 
SUBSTITUTION^ and, because of the Maze Tracing test, SPACE ONE, are more 
heavily speed dependent. 

Laboratory Measures , As with the psychometric data, clusters of 
scores— usually those resulting from a single experiment—were separately 
analyzed to reduce the number of scores to be related to paper and pencil 
test performance. It should also be noted that for the present report 
interest was centered only on speed or time data from the laboratory pro-- 
cedures. Much of the data provided by these S^s will not be considered 
here. 

1 and 2, Binary choice reaction time, Ss fixated a midpoint on a 
CRT display and had to respond by key press to the onset of a character 
display to the left or right of that midpoint. Each £ was run un^^r two 
conditions. In the one -finger condition used index finger of preferred 
hand to depress one of two keys. In the two-finger condition Ss used keys 
under left or right hand respectively to signal detection of display to 
left or right of screen. Data studied here were median response times, 
separately for presentations to the left and right, for the first and 
last (sixth) blocks of 100 presentations under each of the conditions. 
Correlationally, the data clustered by condition and two composite measures 
were developed: 

(1) CHOICE R, TIME, ONE FINGER - Average of standardized scores 
for left and right presentations, first and last block, 

(2) CHOICE R, TIME, TWO FINGER ~ Same average, based on two-finger 
condition, 

3, 4 3 and 5, Delayed auditory feedback study. In this procedure Ss 
were asked to read aloud a page length stream of nonsense syllables under 
three conditions. S^'s reading was taped and played back through ear- 
phones at zero, 150, or 300 millisecond delay. ^Although study of types 
of errors motivated this study, two time scores were of interest here: 
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(3) NO DELAY READING TIME - Time to complete reading of passage 
when feedback was not delayed; 

and 

(4) DELAY READING TIME - Average of standardized times to complete 
reading when feedback was delayed. 

During this same experimental session S^s were set the task of 
repeating back to the experimenter a word replacing R*s with L's and L's 
with R's in so doing. For the present analyses one score was abstracted 
from the resulting data^ 

(5) R/L LATENCY - Median time to transform a word. 

6 and 7. Raven response times . Set I of the Advanced Progressive 
Matrices described earlier was presented one problem at a time and solu- 
tion time measured for each figure. Tliis was done after group adminis- 
tration of Set II and involved 12 problems not contained in the larger set. 
Times for solving earlier and later problems in this set clustered and 
here two scores were employed: 

(6) RAVEN TIME 1 - Average of standardized times to solve problems 
1 and 2; 

and 

(7) RAVEN TIME 2 - Average of standardized times to solve problems 
3 and 9. 

8 and 9o Problem solving time . Another problem solving study called 
for S^s to learn a list of statements which either associated each of a set 
of personal names with an occupation (Ruth is a lawyer) ^ associated these 
same personal names with locations (John is in the pool) ^ or associated 
these locations with activities (If you are in the park you can hear the 
chimes ring). S^s were given a week to learn the list and, following mas- 
tery, each was asked a series of questions to be classified as True or 
False. The questions were such as to require the retrieval of 1^ 2^ 3, or 
4 of the learned facts. Time to answer each question was recorded and the 
two scores of interest here were the parameters of the individual line 
best fitting response times (for correct answers) to the nimiber of facts 
to be retrieved: 

(8) PROBLEM SOLVING SLOPE 
and 

(9) PROBLEll SOLVING INTERCEPT. 
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Results 



Table A reports zero order correlations between the psychometric and 
laboratory measures. Choice reaction times were only weakly related to 
the psychometric measures, and in particular to the speeded measures. 
This contrasts sharply with the Table 1 results for the high school group 
of Study 1. Because of this difference in magnitude of the relations^ the 
stepwise regression analyses of Study 1 were not repeated. Elatherj, a 
series of principal components analyses were completed to ascertain whether 
the relations between psychometric performance and laboratory measures of 
information processing speed would be rendered clearer by the definition 
of broader, composite measure? « 

Table 5 speaks to the relations atoong the selected psychometric 
measures. The two verbal scores had different loadings as did the two 
space measures, the two reasoning subtests.^ and the three perceptual tasks. 
The quantitative score, by this crude index, had inter^test relations not 
unlike those for the second space measure. 

Only one of the rotated components » 4, was very heavily identified 
by speeded measures and might be a perceptual speed indicator. Component 
1 had the appearance of a general cognitive power dimension — high loadings 
by verbal^ quantitative, and spatial ability measures. Component 3 sug- 
gested an accuracy in rule following and Component 2, perhaps, accuracy 
in comparison. 

Relations among the laboratory time measures underlie the principal 
components analysis summarized in Table 6. Component 1 in this analysis 
represented speed (actually, sloimess) of response when the processing 
load was minimal » Component 3 would seem to reflect speed of response 
when there was more pressure on the system—delay of auditory feedback, 
overcoming phonetic convention, searching LTM for learned facts; Component 
4 appeared to associate speed of reading through an LTM store, with speed 
of reading through an external (DAF textjfile. 



Interestingly the speed of solving the figural Raven problems was 
independent of the speed of solving the verbal problems and defined a 
separate Component. The problems differed in other ways, of course; the 
matrix problems became quite difficult and depended for their solution 
upon something other than a closed set of recently acquired bits of 
information. 
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Table 5 

Loadings of Psychometric Measures on Varimax Rotated Components, Study 2 
(Only correlations of .40 or larger indicated. Decimal points omitted. 

(S) denotes a ''speeded" measure.) 



Measure 
Vocabulary 
Grammar 

Quant composite 
Space one (S) 
Space two 
Inference 
Eval argument 
Letter search (S) 
Number checking (S) 
Code substitution 



1 

68 
61 
81 
55 
80 



Component 
2 3 

53 



90 



45 



74 



54 



89 



62 
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Table 6 

a 

Loadings of Laboratory Time Measurias on Varimax Rotated 
Principal Components, Study 2 
(Only correlations of .40 or larger indicated. 
Decimal points omitted.) 



Measure 
Choice time, IF 
Choice time, 2F 
Read time, no delay 
Read time , delay 
R/L latency 
Raven tine 1 
Fiiven time 2 
Problem solving slope 
Problem solving intercept 



1 

71 
93 
42 



Component 
2 3 



89 
81 



83 
68 



64 



81 



67 
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Table 7^ finally, reports the results of a rotated components analysis 
involving both psychometric and laboratory measures. That the number of 
components, G, was equal to the totaL of the number found in the separate 
analyses suggests the two sets did not fit together very neatly. Indeed^ 
four of these eight final components v/ere loaded from only one of the 
sources. Following the* notation, '^Possible Source/' at the bottom of 
Table 1 , the^c corresponded, roughly ^ to laboratory coiuponents 1 (Ll^ 
re^cp-Ciise slomass under minimal load) and 2 time for solving the 

liaven figural problems) and psychometric components 3 (PS^ rule following 
accuracy ras the tentative nane) and 4 perceptual speed). 

The tV/O remaining components from each source appeared to cross with 
each other in this final analysis. Thus the quantitative and space mea- 
sures broke away from th^- verbal in the first psychometric component to 
associate in the fifth component of this second analysis with a reduced 
reading speed (perhaps symptomatic of care in reading). Tae verbal tests 
on the other hand clustered more closely with speed of read'-back when an 
interfering task is surimposed on that read-back. Component 1. Similarly, 
the third laboratory component r,plit into this read-^back under inter- 
ference and an association of overall problem solving time (difference in 
intercept) with number checking or attention to details Component 3- The 
final, shared component. Component 63 was somewhat puzzling. It suggested 
that greater increases in problem solving time associated with retrieving 
an additional bit of information were related to higher scores on the 
inference test. Although not intuitively appealing, one possibility is 
that this component tapped an accuracy (vs. speed) dimension. 

Discussion 

Two of the final components ^ 5 and were suggestive of an 
association betvzeen facility in psychometric problem solving where speed 
is not a critical factor and less rapid information processing in labora-- 
tor^^ problem solving studies. In the other two components of interest, 
1 and 3 5 other processing speeds in the laboratory (DAF; R/L, Intercept) 
v;ere positively associated with accuracy on easier ^ or better learned, 
psychometric tasks. 
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The picture appears somewhat different than in Study 1. There, 
directly measured reaction time to simple stimuli was strongly correlated 
with a host of apparently diverse, psychometric measures. Here, 
response times — from the binary choice study— were lowly correlated with 
psychometric performance and appeared to define a component in isolation 
in the final analysis. 

Study 3 

The differences in findings between the first and second studies 
could have resulted from differences in measurement techniques, from dif- 
ferences in subject sampling, or from other, non-obvious differences between 
studies. The most obvious follow-up was to repeat the measures developed 
in the second study with a subject sample more nearly like those who par^ 
ticipated in the first study. Would stronger relations be found if stu- 
dents at the high school level were used as subjects? 

Me thod 

Subjects. At the end of spring term 1975 high school juniors who had 
completed the W?C battery were recruited through high school counselors. 
Sixty-four volunteers were run in groups of eight for two days apiece. 
Three hour morning sessions were devoted to psychometric testing and 
slightly longer afternoon sessions involved Ss is laboratory data collec- 
tion. S^s were paid at an hourly student rate for participation. 

Measures . Although this more recent high school study involved other 
tasks ^ results will be reported here only on a subset of tasks that were 
in common with Study 1 and, to a greater extent, Study 2. In particular, 
scores were obtained on ten earlier described psychometric instruments: 



1. 


EAS Verbal Comprehension 


2. 


EAS Numerical Ability 


3. 


EAS Numerical Reasoning 


4. 


EAS Verbal Reasoning 


5. 


EAS Symbolic Reasoning 


6. 


Advanced Progressive Matrices (Set II) 


7. 


tlaze Tracing 


8. 


Hidden Figures 


9. 


Two Dimensional Spatial Relations (QIAT) 


10. 


Three Dimensional Spatial Relations (CMAT) 
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Tests were administered under the same conditions as for Study 2 and, where 
applicable. Study 1. 

The Binary Choice Reaction Time task of Study 2 was repeated with the 
high school groups, but in a slightly modified format. Inspection of 
individual records in Study 2 suggested that stabilization was as likely 
after 300 trials as after 600. Ss is this third group, therefore, com- 
pleted only 300 trials (exclusive of an initial block of 30 warm-up 
trials). Also, only the faster two-finger version of the task was employed. 

Results 

Table 8 summarizes the zero order correlations between the binary 
choice reaction time tasks and a set of psychometric scores for this study 
(High School 1975) and for Studies 1 (High School 1973) and 2 (College 
1975), (Data from the two earlier studies are reported here in slightly 
different form: actual choice response times are used from Study 1 
rather than the difference between choice and motor response times used 
in Tables 1-3 and, for Study 2, choice response times are here reported 
separately for the first and last blocks of trials.) The significant re- 
lations of Study 1 were not replicated, for either task, with the college 
sample. For the second, high school group, however, there is a tendency 
for these relations to emerge again, particularly for the Block 3 
response times. 

Means and standard deviations for the three groups on these variables 
are reported in Table 9. In terms of average psychometric performance, 
the second high school group v/as only slightly less proficient than the 
college group. The initial high school group, however, wa^ markedly less 
able. Choice response times are not markedly different between the groups. 
Of greater potential significance, however, are the differences in varia- 
bility between the groups. While there is no consistent pattern of dif- 
ferences on the psychometric instruments the two high school groups are 
appreciably more variable on the choice response time tasks. 

Discussion 

The data for the three groups of subjects taken together provide 
rather clear evidence that the answer to the question of what relations 
exist between choice reaction time and performance on psychometric 
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Table 9 

Means and Standard Deviations, Choice Times and Test Scores, 

Studies 1, 2, and 3 

High school 1973 College 1975 High school 1975 





X 


S.D. 


X 


S.D. 


X 


S.D. 


veroax comprenension 


13.1 


6.0 


19.4 


4.2 


18.6 


4.0 


Numeric ability 


15.8 


7.8 


27.5 


8.3 


26.2 


8.0 


Numeric reasoning 


8.2 


3.8 


13.8 


2.4 


12.5 


2.9 


Verbal reasoning 


13.0 


5.9 


18.6 


3.8 


17.4 


4.8 


Symbolic reasoning 






16.0 


6.3 


14.0 


6.9 


Progressive matrices 






26.5 


4.8 


24.0 


6.4 


Maze tracing 






24.1 


7.7 


26.4 


7.0 


Hidden figures 


3.4 


3.1 


11.2 


6.3 


10.2 


6.9 


Two dimensional space rel'n. 






16.3 


5.0 


13.6 


4.1 


Ttiree dimensional space rel'n. 






15.3 


4.4 


15.7 


5.0 


One finger CRT, initial 


338.0 


83.7 


331.2 


43.8 






One finger CRT^ terminal 






321.3 


37.1 






Two finger CRT, initial 






285.4 


27.0 


303.8 


71.2 


Two finger CRT, terminal 






270.3 


28.6 


288.2 


42.3 
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instruments can only be attempted with a particular population of Ss 
specified. The spread of choice times between individuals seems particu- 
larly influential and it is variability in this performance, not directly 
a basis of selection in college recruitment, that is severely attenuated 
in the entering population of, at least one, state university. It would 
appear that the risk of attenuating information processing relations of 
interest, indeed^ of failing to detect them, is high unless the researcher 
taps a range of performance that is typical of the unselected high school 
population. Narrowing the range of cognitive ability below that may be 
expected to result in studies yielding relations which are complex, and 
difficult to replicate. Using the fuller range, however, holds the pros- 
pect of revealing stronger, more general links between laboratory based 
cognitive performance and mental tests. 
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